Apr 23
21
Google has now added new details that explain the three categories its Google crawlers fall into, they include Googlebot, special-case crawlers and user-triggered fetchers.
Go here to see the original:
Google explains the use cases for its different crawler types